Digital Lib Cover final

This paper presents an expert evaluation of a collaborative annotation system named DiLAS (Digital Library Annotation Service). It can be globally accessed by individuals as well as different user communities, and knowledge is created and shared within English-speaking communities. It contains a collection of textual documents on information science and software science, and it gives access to all kinds of related material such as authors’ home pages, photos, articles, and so on. An analytical evaluation was conducted as a participatory group evaluation, which involved presentation beyond the objectives and rationale papers and development of the prototype. The empirical evaluation of DiLAS consisted of two experiments. The first evaluation was a bottom up evaluation that began at the elementary level with an evaluation of the usability of the interface. A Cognitive Walkthrough approach was chosen using a qualitative approach. The next evaluation moved towards the broader work context with a userand work-centred evaluation involving an entire collaborative task situation, which required knowledge sharing on a common real life work task. This evaluation resulted in a large number of user requirements for the next prototype. World Digital Libraries 1(1): 1–17 A work-centred approach to evaluation of a digital library: annotation system for knowledge sharing in collaborative work ANNELISE MARK PEJTERSEN, Center of Cognitive Systems Engineering, Denmark PREBEN HANSEN, Swedish Institute of Computer Science, Sweden HANNE ALBRECHTSEN, Institute of Knowledge Sharing, Denmark Annelise Mark Pejtersen, Preben Hansen, and Hanne Albrechtsen 2 World Digital Libraries 1(1): 1–17 ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ Introduction Most contemporary digital libraries are repositories of information wherein contents can be searched or browsed by the user. However, when the users’ activity is part of a larger problem solving or work process, searching and browsing of repositories are often not sufficient, since complex problem solving tasks also involve the generation of new idea and knowledge, which often requires collaboration and knowledge sharing of with others. A digital library can support problem solving and work tasks in many different ways. One way is to establish collaborative annotation system that allows the users to work with annotations and share knowledge without losing active support from the digital library. This paper presents a collaborative annotation system named DiLAS (Digital Library Annotation Service). It can be accessed globally by individual user as well as different user communities, and knowledge can be generated and shared globally within Englishspeaking communities, and the content of annotations can also be written in the native language of user groups. It contains a collection of textual documents on information science and software science, and gives access to all kinds of related material such as authors’ home pages, photos, articles, and so on. The implementation of an annotation system in a DLMS changes the characteristics of the application and the behaviour of its users. Writing an annotation may lead the user to a dialogue with the author of a document or the author of an annotation, which eventually can turn into a more elaborated collaboration and creation of additional annotations and new knowledge. For further information on annotation systems, see Agosti M, Benfante L, and Orio N, 2003; Agosti M and Ferro N, 2005a; Agosti M and Ferro N, 2005b; Agosti M, Ferro N, Frommholz I, Thiel U, 2004; Agosti M, Ferro N, and Orio, N, 2005; Andersen H H K, Albrechtsen H, and Cleal B R, 2004; Bottoni P, Civica R, Levialdi S, Orso L, Panizzi E, and Trinchese R, 2004; Bottoni P, Civica R, Levialdi S, Orso L, Panizzi E, and Trinchese R, 2005; Klas C P, Fuhr N, and Schaefer A, 2004; Ovsiannikov I A, McNeill T H, and Arbib M A, 1999. Annotations are supposed to be associated with the documents in digital libraries, and they are regarded as an additional information or as new content of the digital library. Annotation work can also be more or less structured user information on a research topic for which documents are searched and annotations are prepared. Making annotations and talking about them is not an isolated task; on the contrary, it is deeply embedded in the work task. Therefore, the writing of annotations cannot be separated from the actual task to which it is linked. Annotation work and sharing of knowledge may involve knowledge associated with the common work task as well as with the documents in the DLMS. How annotation work is intertwined with work tasks as a means, among others, to accomplish work is very dependent on the work domain and the tasks. In addition to the establishment of a network collaborative annotation system, it is, therefore, necessary to consider how the interplay of sharing knowledge on documents and knowledge on work tasks can be facilitated. Furthermore, this interplay is performed by distributed users with different cognitive capabilities and domain knowledge, different social infrastructures and nationalities – and possibly, different tasks, preferences, and experiences, too. It is important to investigate what are the common ground and the common needs, as well as clarify the importance of the differences and how they can eventually be overcome. A thorough understanding of these issues is only possible through empirical studies, which